Stroke information measuring instrument and stroke information measuring method

ABSTRACT

A stroke information measuring device according to the present invention comprises an imaging section  4  capturing image data of every predetermined time of a swimmer who is swimming, a pixel extracting section  18  extracting pixels corresponding to the body image, a profile data generating section  20  performing the integration of the body image in a predetermined direction and generating the profile data, an end point extracting section  24  extracting the end point of each profile data, a stroke point extracting section  26  extracting the stroke point corresponding to the predetermined posture of the swimmer based on the periodic movement of each end point, and a calculating section  28  calculating the stroke information based on the stroke point.

FIELD OF THE ART

This invention relates to a stroke information measuring device and astroke information measuring method for measuring time required for eachstroke of a swimmer who is swimming and the distance advanced by eachstroke or the like by capturing images of a swimmer.

BACKGROUND ART

In swimming race, a pace-setting of swimmer is one of the importantelements for improving the swimming race ability. There is, therefore, aneed for the measurement of stroke information such as the time(hereinafter, referred to as “stroke time”) required for each stroke ofa swimmer and the traveling distance (hereinafter, referred to as“stroke distance”) by each stroke.

In the measurement of such stroke information, a person captures imagesof a swimmer who is swimming by using a video or the like, and measuresthe stroke information by using a stopwatch while watching the videoimage. In this manner, the conventional measurement of strokeinformation is dependent on manual work.

As a technique for measuring stroke information, J P 2000-42161Adiscloses a technique in that an operator pushes a switch at eachpredetermined phase of the periodic repeated movement of a person (forexample, a swimmer) to be measured while observing the movement of theperson to be measured, and the cycle or the like of the periodicmovement is calculated.

JP H11-159173A discloses a motion capture for capturing the movement ofa swimmer who is swimming.

DISCLOSURE OF THE INVENTION

However, a problem exists in that since the above measurement using thestopwatch is dependent on manual work and spent time and effort, therebytaking much time to feedback the measuring results to the swimmer.

In the technique described in JP 2000-42161A, since the detection of apredetermined phase in the periodic movement of the person to bemeasured is dependent on the operator, measurement accuracy is dependenton the degree of skill of the operator and an error may be caused by arate of reaction or the like. In particular, since each part of the bodyof a swimmer is hidden by water droplets and is dived under water whenswimming, the above error may be larger.

On the other hand, JP H11-159173A discloses a swimming pool fortraining. However, the above publication does not specifically disclosea means for measuring and how measurement performs is unclear.

The present invention has been accomplished in view of the foregoing. Itis an object of the present invention to provide a stroke informationmeasuring device capable of accurately measuring the stroke time andstroke distance of a swimmer who is swimming and in real time and astroke information measuring method.

To achieve the above object, a stroke information measuring deviceaccording to the present invention which captures images of a swimmerduring swimming by using an imaging means and measures the strokeinformation of the swimmer based on the image data obtained, the strokeinformation measuring device comprising: a pixel extracting means forextracting pixels corresponding to the image of the body of a swimmer ineach image data obtained; a profile data generating means for performingthe integration of the pixels corresponding to the image of the bodyextracted by the pixel extracting means in a predetermined direction andgenerating the profile data corresponding to the body image; an endpoint extracting means for extracting an end point of the body imagefrom the profile data; and a stroke point extracting means forextracting a stroke point as the end point corresponding to thepredetermined posture of a swimmer who is swimming based on the periodicmovement of the end point.

Also, a stroke information measuring method according to the presentinvention which measures the stroke information of a swimmer who isswimming, the method comprising the steps of: (1) capturing images of aswimmer during swimming to obtain image data of every predeterminedtime; (2) extracting the pixels corresponding to the body image of aswimmer from each image data obtained; (3) performing the integration ofthe pixels corresponding to the image of the body extracted in apredetermined direction to generate the profile data corresponding tothe image of the body; (4) extracting the end point of the body imagefrom the profile data; and (5) extracting a stroke point as the endpoint corresponding to the predetermined posture of a swimmer who isswimming based on the periodic movement of the end point.

In all of the modern four types of swimming races (free-style,backstroke, breaststroke and butterfly), the swimmer who is swimmingperforms periodic movement having one cycle of one stroke. Therefore,each part of the body is moved periodically according to the stroke.Therefore, by employing the above configuration, if the end pointcorresponding to prescribed posture position of the image of the body isextracted at every predetermined time, the waveform formed by each endpoint changes periodically according to each stroke of the swimmer. Onthe other hand, the end point when the swimmer is in a predeterminedposture for each stroke can be extracted based on the periodic movementof each end point. The information related to the stroke can be analyzedbased on this method.

Therein, the integration of the pixels corresponding to the image of thebody can be performed along the traveling direction of the swimmerand/or the direction vertical to the traveling direction when generatingthe profile data. This is so because the periodic change in thedirection of the body image is large for each swimming stroke, and thedetection thereof is easy.

The swimming-speed of a swimmer is preferably calculated based on themovement of the end point located at the front end in the travelingdirection of a swimmer. This is so because the split time andswimming-speed between the any given positions of a swimmer can bemeasured.

It is preferable that a comparison is performed for each stroke bycalculating at least one of the time for each stroke of a swimmer,traveling distance for each stroke and average swimming-speed for eachstroke based on the stroke point extracted.

The start and the turn can be evaluated by calculating the time requiredfor the start and the traveling distance at the start based on thestroke point previously extracted after the swimmer starts, or bycalculating the time required for a turn and the traveling distance at aturn based on the stroke point extracted just before and just after theturn of the swimmer.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an illustration of the arrangement of an imaging section in astroke information measuring device according to the present invention;

FIG. 2 is a block diagram showing the configuration of the strokeinformation measuring device and related device;

FIG. 3 is a block diagram showing the configuration of an imageprocessor in the device shown in FIG. 2;

FIGS. 4A to 4C are illustrations of H profile data;

FIG. 5 is a D-T map showing the profile data group;

FIG. 6 is a flowchart showing the extracting processing of a strokepoint by a stroke point extracting part;

FIGS. 7A and 7B are illustrations of the profile of a periodic movement;and

FIGS. 8A to 8C are illustrations showing stroke time, stroke distanceand stroke speed.

BEST MODE FOR CARRYING OUT THE INVENTION

Hereinafter, the preferred embodiment of the present invention will bedescribed in detail with reference to the drawings. To facilitate thecomprehension of the explanation, the same reference numerals denote thesame parts, where possible, throughout the drawings, and a repeatedexplanation will be omitted. The scale in the drawings does notnecessarily correspond to that of the description.

FIG. 1 is an illustration of the arrangement of an imaging section 4 ina stroke information measuring device 2 according to the embodiment ofthe present invention. FIG. 2 is a block diagram showing theconfiguration of the stroke information measuring device 2 and a relateddevice.

As shown in FIG. 1, a stroke information measuring device 2 capturesimages of a swimmer 150 who is swimming in a lane 101 of a pool 100 byusing an imaging section 4 (imaging means), and applies image processingto the image data output from the imaging section 4 by using an imageprocessor 6 to measure stroke information such as stroke time and strokedistance. The imaging section 4 comprises fixed cameras arranged on apool side such as a stand, and a plurality of image sections 4 arearranged at an equal interval along the direction of the lane 101 of thepool 100. The image processor 6 is respectively arranged for eachimaging section 4. Thereby, images of the swimmer 150 who is swimmingcan be continuously taken and the stroke information can be continuouslymeasured. The imaging section 4 may be any component which cancontinuously capture images of the swimmer 150 who is swimming such as amoving camera following the travel of the swimmer 150.

The image data output from the imaging section 4 is obtained byA/D-converting NTSC (National Television System Committee) compositevideo signals, and the image data has a size of 640×480, RGB 24 bit. Theimage data is transferred to the image processor 6 for each frame (thatis, every {fraction (1/30)} seconds) in a DMA (Direct Memory Access)mode. At this time, if odd number and even number fields are separatelyprocessed as the image data, time resolution can be improved.

As shown in FIG. 2, the stroke information measuring device 2 isprovided with the above imaging section 4 and the image processor 6. Theimage processor 6 receives the signal showing the official time outputfrom an official timer 8 and corrects the stroke time. The strokeinformation measured by the image processor 6 is output to a integrateddisplay device 10, and is immediately displayed on a display with whichthe integrated display device 10 is provided. The stroke informationmeasured can be stored on a data storage disk 12 via the integrateddisplay device 10.

Next, the configuration of the image processor 6 will be described indetail. FIG. 3 is a block diagram showing the configuration of an imageprocessor 6. The image processor is provided with a memory section 14storing the image data output at each frame from the imaging section 4,a control section 16 controlling the processing timing of the image datastored in the memory section 14, a pixel extracting section 18 (pixelextracting means) extracting the pixel corresponding to the image of thebody of the swimmer in each image data, a profile data generatingsection 20 (profile data generating means) performing the integration ofthe pixel extracted in the direction vertical to the traveling directionof the swimmer and generating the profile data, a profile data groupgenerating section 22 (profile data group generating means) arrangingeach profile data generated in the output order of the image data andgenerating the profile data group.

Further, the image processor 6 is provided with an end point extractingsection 24 (end point extracting means) extracting the end point(hereinafter, referred to as “front end point”) corresponding to the endpart in the traveling direction of the swimmer for each profile data, astroke point extracting section 26 (stroke point extracting means)extracting a front end point when the swimmer is in a predeterminedposture for each stroke as a stroke point from each front end pointbased on the periodic change in the waveform formed by the front endpoint extracted in the profile data group, and a calculating section 28(calculating means) calculating the stroke information of the swimmerbased on the stroke point extracted.

In all of the modern four types of swimming races, (free-style,backstroke, breaststroke and butterfly), the measurement of strokeinformation according to the present invention is based on the periodicmovements of both end parts of the body in the traveling direction of aswimmer who is swimming and both end parts of the body in the directionvertical to the traveling direction for each stroke. Therefore, if theend point corresponding to either end part of the image of the body isextracted at every predetermined time, the end point when the swimmer isin a predetermined posture for each stroke from each end point can beextracted as a stroke point based on the periodic change according toeach stroke of the waveform formed by each end point.

The end point in the traveling direction of the swimmer, that is, frontend point is extracted as the end point corresponding to end part of theimage of the body in the image processor 6 according to the embodiment.This is so because the reference point of the timing of the stroke isgenerally the timing when the head rises highest in case of thebreaststroke and the timing when arm lands in water in other cases.

Each configuration of the abovementioned image processor 6 will bedescribed in detail.

The control section 16 monitors the capturing condition of the imagedata stored in image memory section 14. The control section 16 makes thepixel extracting section 18 start the processing of the image data basedon the signal showing the end of the processing from the calculatingsection 28 when the capturing is finished.

The pixel extracting section 18 executes RGB/HSV (Red, Green, Blue/Hue,Saturation, Value) conversion of all pixels in each image data, make allpixels into binaries of the pixel corresponding to the image of the bodyand other pixels based on hue value, and extracts the pixelcorresponding to the image of the body. The pixel extracting section 18may make all pixels into binaries based on the spectral wavelength andbrightness value.

The profile data generating section 20 performs the integration of thepixel corresponding to the image of the body extracted by the pixelextracting section 18 in the direction vertical to the travelingdirection of the swimmer, and generates the H profile data 30corresponding to the image of the body based on the integration result.FIGS. 4A to 4C are illustrations of the H profile data 30. As shown inFIGS. 4A to 4C, the profile data generating section 20 divides allpixels 32 (see FIG. 4B) performed into each pixel row 32 a along thedirection vertical to the traveling direction of the swimmer 150 (seeFIG. 4A), and calculates the integrated number of pixels composing eachpixel row 32 a. The profile data generating section 20 converts eachpixel row 32 a into one pixel 30 a having a brightness valueproportional to the integrated number, arranges the pixel convertedone-dimensionally, and generates the H profile data 30 corresponding tothe image of the body (see FIG. 4C).

The profile data group generating section 22 arranges each profile data30 generated in the output order of the image data, and generates the Hprofile data group 34 on the D-T map in which the ordinate axisindicates time and abscissa axis indicates distance. FIG. 5 is the D-Tmap showing the H profile data group 34 generated. The D-T map displaysa range of about 15 meters, and the H profile data group 34 shows thatthe swimmer appears from the left on the D-T map, and disappears rightdownward. Each type can be recognized automatically based on adifference in shape due to each type of profile data group generated.

An end point extracting section 24 extracts the end point correspondingto the end point in the traveling direction of the image of the body ofthe swimmer, that is, the front end point P based on the brightnessvalue for each pixel 30 a composing each H profile data 30 that isprovided. As shown in FIG. 4C, the end point extracting section 24extracts the pixel 30 max whose the brightness value is the highest inthe H profile data 30, searches for each pixel 30 a from the pixel 30max extracted to the traveling direction side of the swimmer, andextracts the area between the pixels for which the brightness valuedecreases the most as the front end point P. The end point extractingsection 24 may extract the front end point P as the edge position of thetraveling direction side of the swimmer in the H profile data 30.

A stroke point extracting section 26 extracts the front end point P whenthe swimmer is in a predetermined posture (that is, the posture when thehead rises highest in case of breaststroke and the posture when armlands in water in other cases) for each stroke from each front end pointP as the stroke point P0 based on the periodic change in the waveform(distance locus to time) formed by the front end point P extracted bythe end point extracting section 24 in the D-T map showing the H profiledata group 34. FIG. 6 is a flowchart showing the extracting processingof a stroke point P0 by a stroke point extracting section 26. FIG. 7Aand FIG. 7B are illustrations of the profile of periodic movement.

When the frame number processed from the start of the image capturing tothe present time is f, the average front end point from the present timeto frame number f−n which is n frame earlier is determined by taking theaverage of the front end point in the frames from the present time tothe 2n earlier. The frame number 2n in which the average is taken is setto at least more than the stroke time, and is set to 40 frames in theembodiment. As shown in FIG. 6, FIG. 7A and FIG. 7B, the average frontend point P′ (f-n) in frame number f-n is calculated by (step S50) byextracting the front end point P (f) in the frame number f and by takingthe average of the front end points P in the frame number from thepresent time to the 2n earlier. A difference between the front end pointP (f-n) in the frame number f-n and the average front end point P′ (f-n)is adopted, and the periodic movement profile Q(f-n) is calculated (stepS52). Thereby, the smoothing data Q′ (f-n-1) of Q (f-n-1) in the numberf-n-1 of frames is calculated (step S54) The smoothing data Q′ (f-n-2)calculated and the smoothing data Q′ (f-n-3) are compared, and thesmoothing data Q′ (f-n-2) and the smoothing data Q′ (f-n-1) are compared(step S56). Only when the smoothing data Q′ (f-n-2) is larger than theother, the front end point P (f-n-2) is set as the stroke point P0 (stepS58)

In this manner, a more remarkable periodic change in the waveform can beexhibited by adopting a difference between the front end point P and theaverage front end point P′ and by calculation of the periodic movementprofile Q. Therefore, the stroke point P0 can be accurately extractedbased on the change in curvature of the periodic movement profile Q.

A calculating section 28 calculates the stroke information such as thestroke time and the stroke distance based on each stroke point P0extracted by the stroke point extracting section 26, and outputs to aintegrated display device 10. At this time, since the calculatingsection 28 receives the signal showing the official time output from anofficial timer 8 and corrects the stroke time, accurate strokeinformation can be calculated.

The time required and the distance traveled to extract a first strokepoint P0 after the swimmer starts can be calculated as the time requiredfor the start and the traveling distance from start in the calculatingsection 28. Thereby, the movement of the start of the swimmer, that is,the movement such as diving and submergence can be analyzed. Further,the time required and traveling distance between the stroke points P0extracted just before and just after the turn of the swimmer arecalculated as the time required for turn and traveling distance at aturn. Thereby, the movement of the turn of the swimmer can be analyzed.

Next, the movement of the stroke information measuring device 2 will bedescribed, and a stroke information measuring method according to theembodiment of the present invention will be described.

First, the image data of the swimmer who is swimming, which has beencaptured by the imaging section 4 is output at each frame (that is,every {fraction (1/30)}), and is stored in the memory section 14 of theimage processor 6. The signal showing the start of processing from thecontrol section 16, becomes a trigger, and RGB/HSV conversion of allpixels for each pixel data is executed by the pixel extracting section18. All pixels are made into binaries of the pixel corresponding to theimage of the body and other pixels based on hue value, and the pixelscorresponding to the image of the body are extracted.

The profile data generating section 20 performs the integration of thepixels corresponding to the image of the body in the direction verticalto the traveling direction of the swimmer. The profile data generatingsection 20 generates the H profile data 30 in which the pixels havingthe brightness value proportional to the integrated number are arrangedone-dimensionally. Each H profile data 30 generated is arranged in theoutput order of the image data by the profile data group generatingsection 22, and the H profile data group 34 is generated on the D-T map.

The end point corresponding to the end part in the traveling directionof the image of the body of the swimmer, that is, the front end point Pis extracted based on the brightness value having each pixel 30 acomposing each H profile data 30 by the end point extracting section 24.In the D-T map showing the H profile data group 34, the front end pointP when the swimmer is in a predetermined posture for each stroke isextracted as a stroke point P0 from each end point P by the stroke pointextracting section 26 based on the periodic change in the waveformformed by the front end point P extracted by the end point extractingsection 24.

The stroke information such as the stroke time and the stroke distanceare calculated by the calculating section 28 based on each stroke pointP0, and is output to the integrated display device 10. At this time, inthe calculating section 28, the stroke time is corrected based on thesignal showing the official time output from the official timer 8. Thetime required and the distance advanced to extract a first stroke pointP0 after the swimmer starts are calculated as the time required forstart and the traveling distance at start by the calculating section 28.Further, the time required and traveling distance between the strokepoints P0 extracted just before and just after the turn of the swimmerare calculated as the time required for a turn and traveling distance ata turn.

The front end point when the swimmer is in a predetermined posture foreach stroke can be extracted as a stroke point P0 by the above strokepoint measuring device 2, and the stroke time and stroke distance of theswimmer can be measured accurately and in real time.

The stroke point measuring device 2 extracts the end point correspondingto the end part in the traveling direction of the swimmer as the frontend point P for each H profile data 30. Therefore, the front end point Pis regarded as the front end of the body of the swimmer at the time whenthe profile data corresponds, and the time required for advancing apredetermined distance can be calculated. Therefore, the split time andswimming-speed between any given positions of the swimmer can bemeasured.

The data measured by the stroke point measuring device 2 can be in realtime displayed on the display with which the integrated display device10 is provided by the First Ethernet communication or the like. FIG. 8Athrough FIG. 8C are illustrations showing the stroke time (FIG. 8A) in amen's 200-meter individual medley, stroke distance (FIG. 8B) and strokespeed (FIG. 8C, average swimming-speed for each stroke), and an exampleof the display in the integrated display device 10. A point Pt 1 denotestime required for the start points Pt 2 to 4 denote time required foreach turn, a point Pd 1 denotes traveling distance from the start andpoints Pd 2 to 4 denote traveling distance at each turn.

The data of a plurality of swimmers who are swimming can be displayed onthe integrated display device 10 at the same time. Also, the past dataof the swimmer and the data of other swimmers which are stored in thedata storage disk 12 can be displayed and compared at the same time.Further, the real-time video data is input and the data can beoverlay-displayed.

The embodiment according to the present invention has been described,but it is apparent that the present invention is not limited to theabove embodiments.

For example, in the embodiment, the profile data generating section 20performs the integration of the pixels corresponding to the image of thebody in the direction vertical to the traveling direction of the swimmerto generate the H profile data. However the profile data generatingsection 20 may perform the integration of the pixels in the travelingdirection of the swimmer to generate V profile data.

This is so because the end point of the V profile data periodicallymoves for each stroke and the end point when the swimmer is in apredetermined posture for each stroke from the V profile data as well asthe H profile data can be extracted as a stroke point. If the H profiledata and the V profile data are generated at the same time and thestroke point is extracted from the H profile data and the V profiledata, a more accurate stroke time and stroke distance can be obtained.

INDUSTRIAL APPLICABILITY

The present invention is installed in training facilities or the likefor race swimmers, and is suitable as a stroke information measuringdevice and a stroke information measuring method for evaluating thestroke of the swimmer.

1. A stroke information measuring device which captured images of aswimmer during swimming by using an imaging means and measures thestroke information of the swimmer based on the image data obtained, thestroke information measuring device comprising: a pixel extracting meansfor extracting pixels corresponding to the image of the body of theswimmer in each image data obtained; a profile data generating part forperforming the integration of the pixels corresponding to the image ofthe body extracted by the pixel extracting means in a predetermineddirection and generating the profile data corresponding to the bodyimage; an end point extracting means for extracting an end point of thebody image from the profile data; and a stroke point extracting meansfor extracting a stroke point as the end point corresponding to thepredetermined posture of the swimmer who is swimming based on theperiodic movement of the end point.
 2. The stroke information measuringdevice according to claim 1, wherein the profile data generating meansperforms the integration of the pixels corresponding to the image of thebody extracted by the pixel extracting means along one of the travelingdirection of the swimmer or the direction vertical to the travelingdirection.
 3. The stroke information measuring device according to claim1, wherein the profile data generating means performs the integration ofthe pixels corresponding to the image of the body extracted by the pixelextracting means along both the traveling direction of the swimmer andthe direction vertical to the traveling direction.
 4. The strokeinformation measuring device according to any one of claims 1 to 3,further comprising a swimming-speed calculating means for calculatingthe swimming-speed of the swimmer based on the movement of the end pointlocated at the front end in the traveling direction of the swimmer. 5.The stroke information measuring device according to any one of claims 1to 3, further comprising a stroke information calculating means forcalculating at least one of the time for each stroke of the swimmer,traveling distance for each stroke and average swimming-speed for eachstroke based on the stroke point extracted.
 6. The stroke informationmeasuring device according to claim 5, wherein the stroke informationcalculating means calculates the time required for the start andtraveling distance from the start based on the stroke point previouslyextracted after the swimmer starts.
 7. The stroke information measuringdevice according to claim 5, wherein the stroke information calculatingmeans calculates the time required for a turn and traveling distance ata turn based on the stroke point extracted just before and just afterthe turn of the swimmer.
 8. A measuring method for measuring the strokeof a swimmer during swimming, the method comprising the steps of:capturing images of the swimmer who is swimming to obtain image data ofevery predetermined time; extracting the pixel corresponding to the bodyimage of the swimmer from each image data obtained; performing theintegration of the pixels corresponding to the image of the bodyextracted in a predetermined direction to generate the profile datacorresponding to the image of the body; extracting the end point of thebody image from the profile data; and extracting a stroke point as theend point corresponding to the predetermined posture of the swimmer whois swimming based on the periodic movement of the end point.
 9. Thestroke information measuring method according to claim 8, wherein theintegration of the pixels corresponding to the image of the bodyextracted is performed along one of the traveling direction of theswimmer or the direction vertical to the traveling direction in the stepof generating the profile data.
 10. The stroke information measuringmethod according to claim 8, wherein the integration of the pixelscorresponding to the image of the body extracted is performed along boththe traveling direction of the swimmer and the direction vertical to thetraveling direction in the step of generating the profile data.
 11. Thestroke information measuring method according to any one of claims 8 to10, further comprising the step of calculating the swimming-speed of theswimmer based on the movement of the end point located at the front endin the traveling direction of the swimmer.
 12. The stroke informationmeasuring method according to any one of claims 8 to 10, furthercomprising the step of calculating at least one of the time for eachstroke of the swimmer, traveling distance for each stroke and averageswimming-speed for each stroke based on the stroke point extracted. 13.The stroke information measuring method according to claim 12, furthercomprising the step of calculating the time required for the start andtraveling distance from the start based on the stroke point previouslyextracted after the swimmer starts.
 14. The stroke information measuringmethod according to claim 12, further comprising the step of calculatingthe time required for a turn and traveling distance advanced at the turnbased on the stroke point extracted just before and just after the turnof the swimmer.